ARTICLE
ABSTRACT
We consider division of an empirical distribution of xi, i being the index of a unit, for which we observe xi (e.g., province i, for which xi is the GDP per capita). Values xi are ordered non-decreasingly. We analyse the cumulative distribution, zi = ∑i'=1,...,ixi. The sequence zi is convex. We want to divide the distribution of zi into subsets of i, with the shape of the distribution {zi} possibly well approximated by the segments of the straight line, determined for the subsets, forming a piecewise linear contour, the number of segments being possibly small. This corresponds to the frequently used categorisations for similar distributions (e.g., “developed”, “developing”,... countries). For such categorisations, usually no formal methods are applied but “substantive” prerequisites, or the methods applied are limited to establishing quantiles of the distribution, without considering its shape and the objective premises for determination of a different number of segments, including optimisation of the criterion mentioned before. A general approach is proposed for optimising division of such distribution conform to the criterion mentioned. A general objective function is proposed and its concrete realisation, as well as algorithms. The methodology proposed allows for obtaining the optimum divisions into categories for arbitrary distributions. Yet, on the basis of concrete empirical distributions, problems are outlined, due to the fact that the distributions obtained often display the features, leading to questioning of the foundations of the methodology proposed, and of the very sense of such categorisations. Examples of distributions of this kind, and consequences for the potential categorisations, are discussed. In summary, the methodology proposed, including the criterion function, constitutes a basis for the categorisation with respect to the cumulative distribution, and a tool for evaluating the rationality of the way, in which the distributions are obtained.
KEYWORDS
empirical distribution, optimum division, classes, objective function, bi-partial approach
REFERENCES
[1] Gafner Th., (1991), Mathematical programming approach to classification, Ph. D. dissertation, Institute of Statistics, Faculty of Economics and Business, University of Neuchatel.
[2] Gan G., Ma Ch., Wu J., (2007), Data Clustering, Theory, Algorithms and Applications, SIAM & ASA, Philadelphia.
[3] QOL, (2005), http://www.economist.com/media/pdf/QUALITY OF LIFE.PDF, The Economist Intelligence Unit’s quality-of-life index (as seen on September 25th, 2012).
[4] QOL, (2007), http://www.il-ireland.com/il/qofl07/2007 Quality of Life Index (as seen on September 25th, 2012).
[5] Nielsen L., (2011), Classification of Countries Based on Their Level of Development: How it is Done and How it Could be Done, IMF Working Paper, WP/11/31, IMF.
[6] Owsiński J.W., (1990), On a new naturally indexed quick clustering method with a global objective function, Applied Stochastic Models and Data Analysis, 6, 157-171.
[7] Owsiński J.W., (2011), The bi-partial approach in clustering and ordering: the model and the algorithms, Statistica & Applicazioni, Special Issue, 43-59.
[8] Owsiński J.W., (2012), On dividing an empirical distribution into optimal segments, SIS (Italian Statistical Society) Scientific Meeting, Rome, June 2012, http://meetings.sisstatistica.org/index.php/sm/sm2012/paper/viewFile/2368/229
